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Three  Disciplines: 

Requirements,  Safety,  and  Security 
Engineering 
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Three  Related  Disciplines 


Safety  Engineering 

the  engineering  discipline  within  systems  engineering  concerned  with  lowering 
the  risk  of  unintentional  unauthorized  harm  to  valuable  assets  to  a  level  that  is 
acceptable  to  the  system’s  stakeholders  by  preventing,  detecting,  and  reacting 
to  such  harm,  mishaps  (i.e.,  accidents  and  incidents),  hazards,  vulnerabilities, 
and  safety  risks 

Security  Engineering 

the  engineering  discipline  within  systems  engineering  concerned  with  lowering 
the  risk  of  intentional  unauthorized  harm  to  valuable  assets  to  a  level  that  is 
acceptable  to  the  system’s  stakeholders  by  preventing,  detecting,  and  reacting 
to  such  harm,  misuses  (i.e.,  attacks  and  incidents),  threats,  vulnerabilities,  and 
security  risks 

Requirements  Engineering 

the  engineering  discipline  within  systems/software  engineering  concerned  with 
identifying,  analyzing,  reusing,  specifying,  managing,  verifying,  and  validating 
goals  and  requirements  (including  safety-  and  security-related  requirements) 
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Challenges: 

Combining  Requirements,  Safety,  and 
Security  Engineering 
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Challenges^ 


Requirements  engineering,  safety  engineering,  and  security  engineering 
have  different: 

•  Communities 

•  Disciplines  with  different  training,  books,  journals,  and  conferences 

•  Professions  with  different  job  titles 

•  Fundamental  underlying  concepts  and  terminologies 

•  Tasks,  techniques,  and  tools 
Safety  and  security  engineering  are: 

•  Typically  treated  as  secondary  specialty  engineering  disciplines 

•  Performed  separately  from,  largely  Independently  of,  and  lagging  behind  the 
primary  engineering  workflow: 

(requirements,  architecture,  design,  etc.) 


Software  Engineering  Institute 


Carnegie  Mellon 


Engineering  Safety-  &  Security-Related 
Requirements 

Firesmith/Roberts/Blanchette,  27  April  2010 

©2010  Carnegie  Mellon  University 


Challenges2 


Current  separate  methods  for  performing  requirements,  safety,  and 
security  engineering  are  inefficient  and  ineffective. 

Separation  of  requirements  engineering,  safety  engineering,  and  security 
engineering: 

•  Causes  poor  safety-  and  security-related  requirements  that  are  often: 

—  Vague/unverifiable/unfeasible  architectural  and  design  constraints 

Capabilities  or  goals  rather  than  requirements 

Inadequate  and  too  late  to  drive  architecture  development  and  test 
planning 

•  Makes  it  unnecessarily  difficult  to  achieve  certification  and  accreditation  for 
safe/secure  operations 
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Challenges3 


Poor  requirements  are  a  primary  cause  of  more  than  half  of  all  project 
failures  (defined  in  terms  of): 

•  Major  Cost  Overruns 

•  Major  Schedule  Overruns 

•  Major  Functionality  not  delivered 

•  Cancelled  Projects 

•  Delivered  Systems  that  are  never  used 

Poor  requirements  are  a  major  root  cause  of  many  (or  most)  accidents 
involving  software-intensive  systems. 

Security  ‘requirements’  often  mandated  (e.g.,  Industry  Best  Practices, 
Security  Functions) 

•  Often,  these  are  not  derived  into  meaningful  requirements  at  the  engineering 
level 
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Challenges4 


Constant  tension:  How  safe  and  secure  is  safe  and  secure  enough? 
What  is  needed: 

•  Better  consistency  between  safety  and  security  engineering 

—  More  consistent  concepts  and  terminology 

—  Reuse  of  techniques  across  disciplines 

—  Less  unnecessary  overlap  and  avoidance  of  redundant  work 

•  Better  collaboration: 

—  Between  safety  and  security  engineering 
—  With  requirements  engineering 

•  Better  safety-  and  security-related  requirements 
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Fundamental  Concepts: 

A  Foundation  for  Understanding 
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Quality  Model 
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Quality  Characteristics  (External) 
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Defensibility^ 


Defensibility 

the  quality  characteristic  capturing  the  degree  to  which  the  system: 

•  Properly  prevents,  detects,  reacts  to,  and  adapts  to: 

—  Unintended  and  unauthorized  harm  to  valuable  assets  due  to  the 
occurrence  of 

—  Abuses  enabled  by  the  existence  of 
—  Dangers 

•  Has  defensibility  risks  that  are  acceptably  low  to  its  stakeholders 

•  Valuable  Assets  may  be  people,  organizations,  property,  services,  or 
environments 

•  Harm  may  be  direct  or  indirect,  intentional  or  unintentional,  authorized  or 
unauthorized 
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Defensibility2 


Safety  and  security  aspects 
of  defensibility  are  defined 
in  a  similar  manner  by 
replacing: 


•  Abuse  with  either  mishap  (safety)  or  misuse  (security) 

•  Danger  with  either  hazard  (safety)  or  threat  (security) 

•  Defensibility  risks  with  safety  risks  and  security  risks 
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Safety-  and  Security-Related 
Requirements 
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There’s  More  Than  One  Type 


Too  often,  only  a  single  type  of  requirements  is  considered  when  there  are 
many  types  that  need  consideration: 

•  Special  non-functional  requirements: 

Safety  and  security  requirements  are  quality  requirements 

•  Safety-  and  secunty-significant  requirements  (functional,  data,  and  interface) 

•  Safety  and  security  functions/subsystems  requirements 

•  Safety  and  security  constraints: 

—  Architectural  and  design  constraints 

Mandated  defensibility  controls  (i.e.,  safeguards  and  countermeasures) 

Separation  of  safety/security/requirements  engineering  almost  assures 
gaps  in  requirements 


Gaps  in  Requirements  Lead  to  Shortcomings  in  Delivered 

Systems 
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Four  Types  of  Defensibility-Related  Requirements 


Safety 

Requirements 

Security 

Requirements 


Defensibility 

Requirements 


Safety 

Constraints 

Security 

Constraints 


Defensibility 

Constraints 


Intolerable  Risk 
Requirements 
SAL  =  4 


Defensibility- 
Significant 
Requirements 
SAL  =  1-4 


System 

Requirements 


Primary  Mission 
Requirements 


Supporting 

Requirements 


Requirements 
SAL  =  3 


Moderate  Risk 
Requirements 
SAL  =  2 


Defensibility- 
Independent 
Requirements 
SAL  =  0 


5 


Defensibility  Function  / 
Subsystem  Requirements 


5 


Low  Risk 
Requirements 
SAL  =  1 


Safety/Security 
Assurance  Level 


Safety  Function  /  ■  Security  Function  / 

Subsystem  Requirements  I  Subsystem  Requirements 


(SAL) 
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Example  Safety-  and  Security-Related  Requirements 


Safety  /  Security  Requirement 

“When  in  mode  V,  the  system  shall  limit  the  occurrence  of  accidental  harm  of 
type  W  to  valuable  assets  of  type  X  to  an  average  rate  of  no  more  than  Y  asset 
value  per  Z  time  duration.” 

“When  in  mode  X,  the  system  shall  detect  misuses  of  type  Y  an  average  of  at 
least  Z  percent  of  the  time.” 

Safety  /  Security  Significant  Requirement 

“The  system  shall  automatically  transport  passengers  between  stations.” 

“The  system  shall  enable  users  to  update  their  personal  information.” 

Safety  /  Security  Function  /  Subsystem  Requirement 

“The  system  shall  include  a  fire  detection  and  suppression  subsystem.” 

“The  system  shall  support  the  encryption/decryption  of  sensitive  data.” 


Safety  /  Security  Constraint 


“The  system  shall  not  contain  any  of  the  hazardous  materials  in  Table  X.” 
“The  system  shall  use  passwords  for  user  authentication.” 
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Collaboratively  Engineering 
Safety-  &  Security-Reiated 
Requirements 
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A  Better  Way 


Ensure  close  collaboration  among  Safety,  Security,  and  Requirements 
Teams 

Better  Integrate  Safety  and  Security  Methods: 

•  Concepts  and  Terminology 

•  Techniques  and  Work  Products 

•  Provide  Cross  Training 

Better  Integrate  Safety  and  Security  Methods  with  Requirements  Methods: 

•  Early  during  Development  Cycle 

•  Clearly  define  Team  Responsibilities 

•  Provide  Cross  Training 

Develop  all  types  of  Safety-  and  Security-related  Requirements 
Ensure  that  these  Requirements  have  appropriate  Properties 
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An  Overall  Defensibility  Engineering  Method 
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Defensibility  Analysis  Reqts  Engineering 


Conclusion 
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Summary 


Engineering  safety-  and  security-related  requirements  requires  appropriate 
Concepts  /  Methods  /  Techniques  &  Tools  /  Expertise 

These  must  come  from  the  respective  experts  in; 

•  Requirements  engineering  (safety-  and  security-related  requirements) 

•  Safety  engineering  (analysis  and  safety  goals) 

•  Security  engineering  (analysis  and  security  goals) 

BUT,  Requirements/Safety/Security  Engineering  need  to  be: 

•  Properly  interwoven. 

•  Consistent  with  each  other. 

•  Performed  collaboratively  and  in  parallel  (i.e.,  overlapping  in  time). 

A  collaborative  process  will  advance  Safety  and  Security  Engineering  to  1®^  class 
efforts 


Ultimately,  collaboration  will  improve  the  safety  and 
security  aspects  of  deiivered  systems 
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Defensibility  Quality  Attributes 


I  Occurrence  of  Unauthorized  Harm 

“I 

Occurrence  of  Abuse 
(Mishap,  Misuse,  or  Incident) 

I  Existence  of  External  Abuser 

H 

I  Existence  of  Internal  Vulnerability 

H 

I  Existence  of  Danger  (Hazard  or  Threat) 

H 

I  Existence  of  Defensibility  Risk 

H 

Robustness 


Safety 


Occupational 

Health 


Security 


Survivability 


Problem 

Prevention 

Problem 

Detection 

Problem 

Reaction 

— 

Problem 

Adaptation 

4 


Problem  Type 
Defensibility  Attribute 


I 


Harm  Arrest 


Mitigation 


Recovery 


Analysis 


Counterattack 

(Security) 


Solution  Type 
Defensibility  Attribute 


J 


- 1^  Defensibility  Defensibility  Attribute 


measures 
quality  along  a 


Quality 

Characteristic 


\o 


sz. 


Quality 

Attribute 


is  measured 
along  a 


r 

Quality 

Measurement 

Scale 

Quality 

Measurement 

Method 

6 


defines  the 


Quality 

meaning  or  me 
quality  of  a 

System 

Model 
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Unauthorized  Harm  to  Valuable  Assets 


Stakeholders 


value 


have  an  interest  in  the 


i 


System 


must 

defend 


I 


People 


TV 


I 


Organizations 


Human  Beings 


Roles  Played 


Development 

— 

Owner 

— 

Supplier 

— 

User 

— 

I 


may  occur  to 

I 

■pi  Valuable  Assets  I 

IT 


Property 


I 


Environment 


1 


Services 


Tangible 

Property 


Intangible 

Property 


Private 

Property 


Commercial 

Property 


Software  Engineering  Institute 


Carnegie  Mellon 


Engineering  Safety-  &  Security-Related 
Requirements 

Firesmith/Roberts/Blanchette,  27  April  2010 

©2010  Carnegie  Mellon  University 


Types  of  Harm 
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Types  of  Abuses 
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Types  of  Abusers 


System 

Maintainer 


System 

Developer 


User 


System 

Operator 


Non-malicious 
Human  Abuser 

- 1 - 


Arsonist 


Disgruntled 

Employee 


Cracker 


Identity 

Thief 

- i - 

Mugger 

- i - 

Rapist 

- 1 - 

Foreign 

Government 


Industrial 

Spy 


I 


Professional 

Criminal 


Terrorist 


Non-malicious 
External  System 

I 


5 


Aspect  of  the 
Natural  Environment 


V 


creates 
and  uses 


Attacker 


Non-malicious  Abuser 
(Safety) 

- 1 


Malware 


may  include  existence  of 


5 


Malicious  Abuser 
(Security) 

- 1 - 


Software 

Malware 


M  Spyware 


Hardware 

Malware 


Malware 

System 


1— ^  Backdoor 


Trojan 


H  Worm 


Virus 


Abuser 


is  the 
ultimate 
cause  of  a 


Abuse 


System-External 

Condition 


are  partially  defined 
in  terms  of  the 
existence  of 
system-external  ^ 


may  result  in 


System-Internal 

Condition 


7^ 


Vulnerability 


Defensibility 

Event 


—  Accident  (Safety) 


exploits 


Safety  Incident 


Attack  (Security) 


Security  Incident 
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Vulnerabilities 


Defenses 


Dangers 


eliminate 
or  mitigate 


Abusers 


are  partially 
defined  in  terms  of 
the  existence  of 
system-internal 


exploit 


typically 

cause 


Nonmalicious 

Abusers 


may 


Malicious 

Abusers 


I  cause  ^ 

Abuses 

Vulnerabilities 

n 

I 

may  cause 

Stakeholders 


- r 

have  an 
interest  in  the 


exist 
in  the 


have 


i 


i 


desire 


Unauthorized 

Harm 


System 
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